

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>RBD Exclusive Locks &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="RBD 镜像" href="../rbd-mirroring/" />
    <link rel="prev" title="快照" href="../rbd-snapshot/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../">Ceph 块设备</a> &raquo;</li>
        
          <li><a href="../rbd-operations/">Ceph 块设备运维</a> &raquo;</li>
        
      <li>RBD Exclusive Locks</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/rbd/rbd-exclusive-locks.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../">
          

          
            
            <img src="../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 块设备</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../rados-rbd-cmds/">基本命令</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../rbd-operations/">运维</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../rbd-snapshot/">快照</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">互斥锁、独占锁</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#blocklisting">Blocklisting</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-mirroring/">镜像</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-live-migration/">实时迁移</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-persistent-read-only-cache/">永久只读缓存</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-persistent-write-log-cache/">永久写日志缓存</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-encryption/">加密</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-config-ref/">配置选项 (librbd)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-replay/">RBD 重放</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../rbd-integrations/">对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../man/">手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/">APIs</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="rbd-exclusive-locks">
<span id="id1"></span><h1>RBD Exclusive Locks<a class="headerlink" href="#rbd-exclusive-locks" title="Permalink to this headline">¶</a></h1>
<p id="index-0">Exclusive locks are a mechanism designed to prevent multiple processes
from accessing the same Rados Block Device (RBD) in an uncoordinated
fashion. Exclusive locks are heavily used in virtualization (where
they prevent VMs from clobbering each others’ writes), and also in RBD
mirroring (where they are a prerequisite for journaling).</p>
<p>Exclusive locks are enabled on newly created images by default, unless
overridden via the <code class="docutils literal notranslate"><span class="pre">rbd_default_features</span></code> configuration option or
the <code class="docutils literal notranslate"><span class="pre">--image-feature</span></code> flag for <code class="docutils literal notranslate"><span class="pre">rbd</span> <span class="pre">create</span></code>.</p>
<p>In order to ensure proper exclusive locking operations, any client
using an RBD image whose <code class="docutils literal notranslate"><span class="pre">exclusive-lock</span></code> feature is enabled should
be using a CephX identity whose capabilities include <code class="docutils literal notranslate"><span class="pre">profile</span> <span class="pre">rbd</span></code>.</p>
<p>Exclusive locking is mostly transparent to the user.</p>
<ol class="arabic simple">
<li><p>Whenever any <code class="docutils literal notranslate"><span class="pre">librbd</span></code> client process or kernel RBD client
starts using an RBD image on which exclusive locking has been
enabled, it obtains an exclusive lock on the image before the first
write.</p></li>
<li><p>Whenever any such client process gracefully terminates, it
automatically relinquishes the lock.</p></li>
<li><p>This subsequently enables another process to acquire the lock, and
write to the image.</p></li>
</ol>
<p>Note that it is perfectly possible for two or more concurrently
running processes to merely open the image, and also to read from
it. The client acquires the exclusive lock only when attempting to
write to the image. To disable transparent lock transitions between
multiple clients, it needs to acquire the lock specifically with
<code class="docutils literal notranslate"><span class="pre">RBD_LOCK_MODE_EXCLUSIVE</span></code>.</p>
<div class="section" id="blocklisting">
<h2>Blocklisting<a class="headerlink" href="#blocklisting" title="Permalink to this headline">¶</a></h2>
<p>Sometimes, a client process (or, in case of a krbd client, a client
node’s kernel thread) that previously held an exclusive lock on an
image does not terminate gracefully, but dies abruptly. This may be
due to having received a <code class="docutils literal notranslate"><span class="pre">KILL</span></code> or <code class="docutils literal notranslate"><span class="pre">ABRT</span></code> signal, for example, or
a hard reboot or power failure of the client node. In that case, the
exclusive lock is never gracefully released. Thus, when a new process
starts and attempts to use the device, it needs a way to break the
previously held exclusive lock.</p>
<p>However, a process (or kernel thread) may also hang, or merely lose
network connectivity to the Ceph cluster for some amount of time. In
that case, simply breaking the lock would be potentially catastrophic:
the hung process or connectivity issue may resolve itself, and the old
process may then compete with one that has started in the interim,
accessing RBD data in an uncoordinated and destructive manner.</p>
<p>Thus, in the event that a lock cannot be acquired in the standard
graceful manner, the overtaking process not only breaks the lock, but
also blocklists the previous lock holder. This is negotiated between
the new client process and the Ceph Mon: upon receiving the blocklist
request,</p>
<ul class="simple">
<li><p>the Mon instructs the relevant OSDs to no longer serve requests from
the old client process;</p></li>
<li><p>once the associated OSD map update is complete, the Mon grants the
lock to the new client;</p></li>
<li><p>once the new client has acquired the lock, it can commence writing
to the image.</p></li>
</ul>
<p>Blocklisting is thus a form of storage-level resource <a class="reference external" href="https://en.wikipedia.org/wiki/Fencing_(computing)">fencing</a>.</p>
<p>In order for blocklisting to work, the client must have the <code class="docutils literal notranslate"><span class="pre">osd</span>
<span class="pre">blocklist</span></code> capability. This capability is included in the <code class="docutils literal notranslate"><span class="pre">profile</span>
<span class="pre">rbd</span></code> capability profile, which should generally be set on all Ceph
<a class="reference internal" href="../../rados/operations/user-management/#user-management"><span class="std std-ref">client identities</span></a> using RBD.</p>
</div>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="../rbd-mirroring/" class="btn btn-neutral float-right" title="RBD 镜像" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../rbd-snapshot/" class="btn btn-neutral float-left" title="快照" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>